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Abstract 



This study focused on the relationship between high school students' course- 
taking patterns and their achievement of higher-order thinking skills. PLAN scores (in 
grade 10) and ACT Assessment scores (in grade 11 or 12) were collected for 73,818 
students in 1,174 high schools. The findings showed that, in a typical high school, 11th- 
and 12th-grade students who took upper-level mathematics or science courses had 
higher ACT Mathematics, Science Reasoning, and Composite scores than those who did 
not take these courses, regardless of their previous PLAN scores, gender, family income 
level, and ethnicity. Further, average score differences between males and females, and 
between Caucasian- Americans and ethnic minorities were reduced when these variables 
were considered. The effects of taking mathematics and science courses were relatively 
small for students attending schools in districts where per-pupil expenditures were low 
and the percentage of ethnic minority students was high. 
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Factors Associated with Longitudinal Educational Achievement, 

As Measured by PLAN and ACT Assessment Scores 

The courses that students take in high school, the quality of the instruction and 
counseling they receive, their motivation, and their backgrounds all affect their 
educational achievement. Examining the relationships among test scores, high school 
course-taking experiences, and student and high school characteristics can help provide 
supportive evidence for the use of test scores as tools for monitoring educational 
development. Moreover, such research improves our understanding of the effects of 
different schools on student achievement. 

The ACT Assessment and PLAN tests (ACT, 1989, 1992, 1995a) are intended to 
measure higher-order thinking skills in four content areas (English, Mathematics, 
Reading, and Science Reasoning). The PLAN tests are used in educational and career 
planning of high school sophomores. The ACT Assessment tests are used in college 
admissions and course placement. Both assessments include course work information 
and student information sections. 

Related research (e.g., ACT, 1994) consistently highlights discrepancies in 
educational achievement favoring students who have completed a college preparatory 
"core" of courses, compared to those who have not. This core of courses, as defined by 
ACT, includes four years of English, and three years each of mathematics, social studies, 
and natural sciences. Large differences in average achievement scores have been found 
favoring PLAN/ACT-tested students who indicated that they intended to complete, or 
had completed, college preparatory core course work, compared to those who did not 
(ACT, 1993). 
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Other studies have investigated student performance on either PLAN or the ACT 
Assessment and its relationship to course work taken or planned in high school (Noble 
& Powell, 1995; Noble, Crouse, Sawyer, & Gillespie, 1992; Noble, 1990; Noble & 
McNabb, 1989). The results supported the claim that students who have taken, are 
currently taking, or are planning to take college-preparatory course work not only have 
higher test scores but are better prepared academically for college. However, other than 
the 1993 ACT research noted above, all the research conducted to date concerning 
differential course-taking patterns and subsequent achievement, as measured by PLAN 
and ACT scores, has been based on data from students who had taken one test or the 
other, but not both. This study focused on cohort achievement and specific course-taking 
patterns using longitudinal student data (i.e., students had taken both PLAN and the 
ACT Assessment). 

Because high school course requirements for graduation vary from state to state 
and across school districts, not all students who aspire to attend a postsecondary 
institution take three or four years of college preparatory courses in a subject area. 
Although these students may accumulate required credits for graduation in these subject 
areas, grouping their PLAN and/or ACT scores with those of students who have taken 
a college preparatory sequence of courses distorts overall achievement results. 
Consequently, achievement trends were investigated in terms of specific courses and 
patterns of course work taken, in order to understand the course-taking patterns within 
and across subject areas that contribute most to student achievement. Two specific 
questions were investigated: 
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• What are the relationships among courses taken, grades received, high 
school attended, ethnicity, gender, family income, and educational 
achievement, as measured by PLAN and ACT Assessment scores? 

• How useful are courses taken, grades received, ethnicity, gender, and 
family income in modeling academic achievement, as measured by ACT 
scores, while statistically controlling for preexisting differences in academic 
achievement, as measured by PLAN scores? 

The answers to these questions provide evidence for using PLAN and ACT Assessment 
scores as measures of student achievement and as tools for monitoring educational 
effectiveness. 

Data 

The PLAN /ACT cohort file for the graduating class of 1994 contained matched 
records of students who completed PLAN during their sophomore year (fall, 1991) and 
the ACT Assessment during their junior or senior year, prior to graduating in 1994. 1 If 
students took the ACT Assessment on more than one occasion, only the most recent 
ACT Assessment record was used. Each student record consisted of PLAN and ACT 
Assessment scores (i.e., English, Mathematics, Reading, Science Reasoning, and 
Composite scores), student background information (gender, family income, 
majority /minority ethnic group membership), and self-reported course work information 
(course work taken, course grades [ACT Assessment only]). Dummy-coded variables 



’Virtually every student in the total matched file attended the same high school at the time of ACT 
testing that he or she had attended at the time of PLAN testing. There were some students (slightly 
less than 1%), however, who transferred to different schools prior to taking the ACT Assessment. 
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were created for gender (males = 0; females = 1), majority /minority ethnic group 
membership (Caucasian- American/ White = 1; African-American /Black or Hispanic = 0), 
and course work taken (taken or currently taking = 1; not taken = 0). An ordered 
categorical variable was created for family income (less than $11,999 = 1; $12,000-$23,999 
= 2; $24,000-35,999 = 3; $36,000-$49,999 = 4; $50,000 or more = 5). 

The total matched file contained 191,025 student records from 3,088 high schools 
across the nation, with a minimum of 25 records per school. Of these, 73,818 student 
records from 1,174 high schools had valid values for all relevant independent and 
dependent variables. These student records were used in the analyses. 

School and district demographic information from the Market Data Retrieval 
(MDR) data file was appended to each student record and included school location 
(urban/suburban/rural), district total per pupil expenditure, and district percentage of 
students below poverty level. The primary purpose for including these variables was 
to describe schools. 

From information provided in the PLAN and ACT data files, variables that 
reflected students' course-taking patterns and grade averages were created. These 
variables were based on previous research (Noble, et al., 1992; Noble, 1990), and were 
used to address the chief issues in this investigation: the relationships among student 
course-taking patterns, course grades, and PLAN and ACT Assessment scores. 

Method 

Means and standard deviations were computed for PLAN scores, ACT scores, 
course work and grade information, and student and high school characteristics. These 
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statistics were computed for each high school; the statistics were then summarized across 
schools using minimum, median, and maximum values. For comparative purposes, 
means and standard deviations based on data pooled over institutions were also 
computed for all relevant variables. 

Correlation coefficients between ACT Assessment scores and relevant independent 
variables were calculated based on data pooled across all schools. Correlations between 
pairs of PLAN scores and ACT scores were fairly large, ranging from .69 (Science 
Reasoning) to .88 (Composite). These statistics are reported in Appendix A. Partial 
correlation coefficients were also computed between ACT scores and other independent 
variables after the effect of PLAN scores was removed. Those predictors with the 
strongest, statistically significant (p < .001) correlations with ACT Assessment scores 
were included in regression models for predicting ACT Assessment scores. 



Preliminary regression models were developed for PLAN and ACT scores, using 
data pooled across all schools. In these models, ACT scores were regressed on logically- 
related PLAN scores (e.g., ACT Mathematics score was regressed on PLAN Mathematics, 
Science Reasoning, and Composite scores, but not on PLAN English score). The models 
were evaluated in terms of model statistical significance (p < .001), multiple R, standard 
error of estimate (SEE), and the statistical significance (p < .001) of the regression 
coefficient associated with each independent variable. In addition, for models that 
contained two or more PLAN scores as independent variables, only statistically 



PLAN/ACT Regression Models 




ER AC 



6 



significant, noncollinear PLAN scores were considered. Because of the large sample size, 
most regression coefficients were statistically significant for all of the models tested. 

Models containing more than one PLAN score showed minimal increases in 
accuracy over the single independent variable models, as measured by multiple R and 
SEE. Models containing either the PLAN Composite score or Mathematics score as the 
independent variable were determined to be optimal for modeling all five ACT scores. 
These models are described below: 



Test Score Models (N = 191,025) 



Dependent variable 


Independent variable 


r 


SEE 


ACT Composite 


PLAN Composite 


.88 


2.15 


ACT English 


PLAN Composite 


.82 


2.91 


ACT Reading 


PLAN Composite 


.79 


3.68 


ACT Mathematics 


PLAN Mathematics 


.82 


2.82 


ACT Science Reasoning 


PLAN Composite 


.77 


2.95 



Curvilinear relationships between ACT and PLAN scores were investigated for 
the test score models by including second- and third-order polynomial terms. Although 
these terms were statistically significant (p < .001) for all of the models tested, multiple 
R was only slightly larger than that of models containing only a first-order polynomial 
term. The one-variable models were therefore used in subsequent analyses. 

Additional Independent Variables 

Although students' PLAN scores were the principal independent variables for 
modeling ACT Assessment scores, other potential independent variables were also 
investigated. This was done using stepwise selection. Additional independent variables 
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were retained in the models only if (1) they were statistically significant (p < .001), over 
and above PLAN scores, (2) they resulted in a practical (i.e., 5%) increase in multiple R 
or decrease in SEE, and (3) they were not collinear with other independent variables. 
Because of the likelihood of misspecified models, the results of the stepwise selection 
method were confirmed by examining the simple correlations among the independent 
variables and ACT scores, and by determining whether the relationships were 
reasonable. For example, English course work taken may be positively associated with 
ACT Mathematics score, but such an association may not be as meaningful for 
explaining student achievement as is the association between mathematics course work 
and Mathematics score. 

The selection of additional independent variables proceeded in the following 
manner: Blocks of independent variables were entered in models containing PLAN score 
(either Mathematics or Composite) as the only independent variable. Stepwise selection 
was then performed within blocks, where applicable. The order of entry (see A, B, & 
C in table below) of the blocks was varied to ensure that their contribution to explaining 
ACT Assessment performance was clearly understood, as shown below: 
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Blocks of Predictors and Their Order of Entry 







Order of entry 
for block 


Block 


Description 


A 


B 


C 


Grade level/ time 
variables 


Educational level at the time of ACT 
Assessment testing, length of time 
between taking PLAN and the ACT 
Assessment 


1 


3 


2 


Student background 
variables 


Gender, family income, ethnicity 


2 


1 


4 


Aspiration variables 


Change over time in college-bound 
status, expressed need for help 


3 


4 


3 


Course work/ grade 
variables 


Courses taken and grades earned in 
those courses 


4 


2 


1 



Note: PLAN Mathematics or Composite scores were included in all models. 



When stepwise regression analyses were performed on each separate block, no 
meaningful improvement in multiple R (5% increase) or SEE (5% decrease) was found 
as a result of including course work and grade variables, grade level and time variables, 
student background variables, or aspiration variables in the test score models. For 
example, when ACT Mathematics was regressed on PLAN Mathematics, the simple 
correlation between these variables was .81 (n = 73,818). When course grades for algebra 
II, geometry, and trigonometry, and variables indicating whether biology, chemistry, and 
physics were taken were added to this model, the increase in multiple R was only .03 
(R = .84). Similar findings occurred for ACT English, Reading, Science Reasoning, and 
Composite models when selected course work variables (e.g., grade averages, number 
of courses taken, weighted grades in highest level courses taken) were added. 
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Final Model Development 

Because multiple-independent variable models yielding meaningful increases in 
multiple R and/or decreases in SEE over the test score models could not be identified, 
an alternative, explanatory approach to model development was chosen. This approach 
capitalized on the fact that regression coefficients associated with course work variables 
could be used to describe the unit change in ACT score resulting from taking a 
particular course, while holding PLAN scores and other relevant variables constant. 
Because the course work variables were dichotomous, they could be interpreted as 
reflecting the difference in average ACT score associated with taking a particular course, 
while statistically controlling for differences in PLAN score, gender, family income, 
majority /minority ethnic group membership, and the other course work taken. For 
example, a regression coefficient of 1.5 for the independent variable trigonometry in the 
ACT Mathematics model indicates that when other differences were statistically 
controlled, the average ACT Mathematics score for students who took trigonometry was 
1.5 scale score units higher than the average ACT Mathematics score for students who 
did not take trigonometry. 

Course Work Taken 

For the ACT Mathematics, Science Reasoning, and Composite score models, we 
focused on courses taken in mathematics and science. The relationships between these 
courses and ACT Mathematics and Science Reasoning scores were logical and easily 
interpreted. In addition, mathematics and science courses taken were more strongly 
correlated with these ACT scores than were other courses taken. English and social 
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studies course work taken contributed only minimally, if at all, to multiple R for ACT 
English and Reading models, over and above PLAN Composite score. We therefore 
included only PLAN Composite score and student demographic characteristics as 
independent variables in these models. 

High School Attended 

Because PLAN scores were to be included as covariates (to control for preexisting 
differences in the academic achievement of sophomores), including high school as an 
effect-coded dummy variable in the models was contingent upon satisfying the 
(ANCOVA) assumption of homogeneous slopes and residual variances across high 
schools. If heterogeneous slopes were found, then this would require fitting a separate 
regression equation for each school. 

To test the assumption of homogeneity of slopes, high school attended by PLAN 
score interactions were included in an ANCOVA model containing PLAN score and high 
school main effects. Three random samples of 20 high schools each, and two random 
samples of 480 high schools each were drawn for the ANCOVA analysis 2 . Effect sizes 
for the interactions were computed as the range (maximum - minimum) of the 
regression coefficients divided by their average standard error. The effect sizes were 
sufficiently large to conclude that the slopes differed across high schools. 

Because the assumption of homogeneous slopes was not met, within-school 
regression equations were developed. These equations utilized a common set of 
independent variables, and relevant within-school regression statistics (e.g., multiple R, 

2 Random samples of high schools were drawn because including all 3,088 high schools in the 
analysis exceeded computing resources. 
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SEE, regression coefficients) were summarized across schools using minimum, median, 
and maximum values. Variability in regression statistics across schools would help 
illustrate the effect of high school attended on educational achievement. The final 
models were: 



ACT English = / (PLAN Composite; gender; family income; 

majority/minority ethnic group membership) 



ACT Mathematics = 



ACT Reading 



ACT Sci. Reas. 



ACT Composite = 



/ (PLAN Mathematics; trigonometry, calculus, 
chemistry, and physics course work taken; gender; 
family income; majority /minority ethnic group 
membership) 

/ (PLAN Composite; gender; family income; 
majority/ minority ethnic group membership) 

/(PLAN Composite; trigonometry, calculus, chemistry, 
and physics course work taken; gender; family 
income; majority/ minority ethnic group membership) 

/(PLAN Composite; trigonometry, calculus, chemistry, 
and physics course work taken; gender; family 
income; majority /minority ethnic group membership) 



To further assess the effect of high school attended on students' ACT Assessment 



performance, given their PLAN scores and demographic variables, each high school was 
classified into one of five categories of expected ACT Assessment performance. High 
schools with predicted mean ACT Composite scores that differed by ± 2 score units from 
the pooled predicted mean ACT Composite score across all students and schools were 
categorized as performing above expectation (+ 2 or more score units) or below 
expectation (-2 or fewer score units). To be classified as performing above or below 
expectation, predicted school means were also required to be statistically significantly 
different (p < .001) from the pooled predicted mean ACT Composite score. A 
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description of this method is provided in ACT (1995b). Additional performance 
categories were also identified: performance slightly above expectation (+1 or more score 
units, but less than +2 score units), performance slightly below expectation (-1 or fewer 
score units, but no fewer than -2 score units), and performance as expected (between -1 
and +1 score units). Schools with predicted means that were not statistically 
significantly different from the pooled predicted mean ACT Composite score were 
placed in the category of performance as expected, regardless of the difference in score 
units. Regression statistics and school characteristics were summarized across high 
schools within school performance category. 

Results 

Descriptive statistics, summarized across schools, for PLAN scores, ACT 
Assessment scores, course work taken, and student background variables are shown in 
Table 1. Minimum, median, and maximum values summarize the distribution of 
statistics (mean or percentage) across the 1,174 high schools that had at least 25 students 
with valid values on the relevant variables (n = 73,818). The median value represents 
the mean observed in the typical high school. 

Mean PLAN scores typically ranged from 19.2 (Reading) to 20.0 (English) across 
schools; for the ACT Assessment, typical means ranged from 21.6 (Mathematics) to 23.0 
(Reading). In general, the ACT Assessment and PLAN score means summarized across 
high schools were very similar to the corresponding means based on pooled data, which 
are shown in Appendix B. 
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TABLE 1 

Distribution, Across Schools, of Descriptive Statistics for PLAN 
Scores, ACT Assessment Scores, and Independent Variables 
(Number of schools = 1174) 



Statistic 


Variable 


Med. 


Min. 


Max. 


Mean 


PLAN 

English 


19.9 


13.2 


24.2 




Mathematics 


19.3 


14.3 


22.9 




Reading 


19.2 


12.3 


23.3 




Science Reasoning 


19.3 


15.1 


22.2 




Composite 


19.5 


13.8 


23.1 




ACT Assessment 
English 


22.3 


15.5 


27.3 




Mathematics 


21.6 


15.8 


27.1 




Reading 


23.0 


15.3 


29.3 




Science Reasoning 


22.4 


16.3 


27.0 




Composite 


22.5 


16.1 


27.7 


Percent 


Course work taken 

Trigonometry 


48 


3 


98 




Calculus 


12 


1 


59 




Chemistry 


86 


23 


100 




Physics 


29 


2 


96 




Gender 

Female 


59 


1 


99 




Male 


41 


1 


99 




Family income 

Less than $11,999 


5 


0 


62 




12,000 - 23,999 


11 


1 


44 




24,000 - 35,999 


19 


2 


58 




36,000 - 49,999 


21 


2 


50 




50,000 or more 


40 


2 


89 




Majority/ minority 
Majority 


94 


2 


100 




Minority 


6 


1 


98 



Median percentages of course work taken ranged from 12% (calculus) to 86% 
(chemistry). Across schools, the typical percentage of females (59%) was larger than the 
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percentage of males, the median percentage of students reporting a family income of 
$50,000 was 40%, and the median percentage of students indicating that they were ethnic 
minorities was 6%. These percentages were similar to those based on pooled data (see 
Appendix B). 

Student and School Performance Differences 
Regression Statistics Summarized across All Schools 

Distributions of student sample sizes, multiple Rs, standard errors of estimate 
(SEEs), and (unstandardized) regression coefficients are shown, by ACT Assessment test 
score model, in Tables 2a and 2b. Table 2a includes results for ACT Mathematics, 
Science Reasoning, and Composite scores. Table 2b includes results for ACT English and 
Reading. The English and Reading models are presented separately because they did 
not include course work taken variables. 

Table 2a shows, for example, that ACT Mathematics score was modeled as a 
function of PLAN Mathematics score, gender, family income, majority /minority ethnic 
group membership, and whether or not trigonometry, calculus, chemistry, and physics 
were taken. Student sample sizes, across schools, ranged from 25 to 380 for the ACT 
Mathematics model (see the first row of numbers in Table 2a). The median multiple R, 
across schools, for the ACT Mathematics model was .87, with a median SEE of 2.44. 
Median regression coefficients for this model ranged from -.64 (gender) to 1.37 
(trigonometry). Median multiple Rs for all models were fairly large, ranging from .79 
(Reading, see Table 2b) to .91 (Composite). Median SEEs ranged from 1.94 (Composite) 
to 3.59 (Reading). 
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TABLE 2a 

Distributions, Across Schools, of Regression Statistics 
for Modeling ACT Mathematics, Science Reasoning, and Composite Scores 

(Number of schools = 1174) 



ACT Assessment score 







Mathematics 


Science Reasoning 




Composite 




Statistic 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


N 1 


49 


25 


380 


; 49 


25 


380 


49 


25 


380 


R 


.87 


.55 


.98 


.82 


.42 


.95 


.91 


.63 


.97 


SEE 


2.44 


1.11 


3.64 


2.70 


1.16 


4.15 


1.94 


1.15 


3.25 


Intercept 

Regression coefficients 


4.10 


-11.02 


18.70 


3.46 


-8.62 


24.21 


.71 


-10.21 


14.19 


PLAN 

Mathematics 

/Composite 2 


.80 


.12 


1.53 


.92 


.21' 


1.68 


1.04 


.57 


1.45 


Trigonometry 


1.37 


-4.41 


7.80 


.43 


-6.59 


5.39 


.62 


-3.26 


4.85 


Calculus 


1.27 


-7.34 


10.50 


.19 


-8.91 


7.52 


.50 


-5.53 


5.07 


Chemistry 


.87 


-8.07 


8.05 


.66 


-8.15 


8.17 


.61 


-5.03 


5.84 


Physics 


.97 


-5.57 


6.87 


.56 


-8.45 


8.60 


.49 


-3.06 


7.23 


Gender 


-.64 


-4.91 


3.17 


-1.51 


-4.86 


2.45 


-.60 


-3.55 


2.10 


Family income 


.03 


-.69 


.90 


.02 


-.72 


.68 


.04 


-.65 


1.02 


Majority /minority 


.42 


-8.24 


8.72 


.67 


-8.01 


11.93 


.48 


-5.72 


7.14 



= number of students. 

2 PLAN Mathematics score was used to model ACT Mathematics score. PLAN Composite score was used to model 
ACT Science Reasoning and Composite scores. 
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TABLE 2b 



Distributions, Across Schools, of Regression Statistics 
for Modeling ACT English and Reading Scores 
(Number of schools = 1174) 



Statistic 


ACT Assessment score 


Med. 


English 

Min. 


Max. 


Med. 


Reading 

Min. 


Max. 


N 1 


49 


25 


380 


49 


25 


380 


R 


.83 


.50 


.94 


.79 


.44 


.94 


SEE 


2.76 


1.73 


4.28 


3.59 


2.15 


5.47 


Intercept 


-2.24 


-17.30 


10.60 


-3.92 


-19.54 


11.26 


Regression coefficients 














PLAN Composite 


1.18 


.64 


1.74 


1.36 


.67 


2.01 


Gender 


.53 


-2.52 


3.84 


-.11 


-5.25 


4.47 


Family income 


.08 


-.78 


1.00 


.03 


-1.03 


1.16 


Majority / minority 


.48 


-7.06 


8.46 


.36 


-10.55 


10.46 



*N = number of students. 



The course work regression coefficients reflect the average difference in ACT score 
associated with taking or not taking a particular course. When differences in PLAN 
score, gender, majority /minority ethnic group membership, and family income were 
statistically controlled, students who took trigonometry, calculus, chemistry, or physics 
typically showed higher average ACT scores than students who did not take the courses. 
The typical (median) difference in mean ACT score between students taking and not 
taking a particular course ranged from .19 scale score units (on ACT Science Reasoning 
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for calculus) to 1.37 scale score units (on ACT Mathematics for trigonometry). Mean 
score differences were largest for ACT Mathematics 3 . 

Negative course work regression coefficients that exceeded the respective standard 
errors of measurement for the ACT Assessment tests were found for about six percent 
of the schools, indicating a higher mean ACT score for students not taking a particular 
course 4 . In many instances, these coefficients were associated with relatively small 
within-school sample sizes. Negative coefficients were also found for some schools in 
which very small percentages (e.g., 2%) of students had taken a particular mathematics 
or science course. One possible explanation for this finding is that a school's more 
academically able students (i.e., those who earn high ACT scores) might elect not to take 
a particular course, for whatever reason. For example, a large percentage of students 
who took trigonometry at a particular school might, because of a teacher's reputation, 
choose not to take calculus. In spite of this, these students might earn high ACT 
Mathematics scores, resulting in an inverse relationship between calculus course taking 
and test score. Of course, it is possible that the negative regression coefficients 
accurately reflect the relationship between ACT scores and course taking at some 
schools. Regardless of the reasons for negative regression coefficients occurring for some 



<1 

When examining course work regression coefficients, it is important to consider that the scale score units for each test are 
not equivalent. The scales were not constructed to ensure that a particular scale score on one test is comparable to the same 
scale score on another test (e.g., a 21 on the Mathematics test is not comparable to a 21 on the Science Reasoning test). Although 
mean score differences were largest for ACT Mathematics, this does not necessarily mean that the effect of course taking was 
greatest for this test, relative to effects for the other tests. 



The standard errors of measurement for ACT Mathematics, Science Reasoning, and the Composite are approximately 1.5, 



2, and 1 scale score units, respectively. 




• \ ^ 



23 




18 



schools, their effect on the findings is likely minimal, given the large number of schools 
in this study. 

Gender. The regression coefficients associated with gender reflect the typical 
adjusted mean difference in ACT scores between males and females, when all other 
variables were statistically controlled. Note that the sign (-) of the regression coefficient 
for gender simply reflects its arbitrary coding (females = 1, males = 0). Males, on 
average, had higher mean ACT Mathematics, Science Reasoning, and Composite scores 
than did females when PLAN score, course work, family income, and majority /minority 
ethnic group membership were statistically controlled. The Reading scores of males 
were also higher than those of females when PLAN score, family income, and 
majority /minority ethnic group membership were statistically controlled. Median 
average score differences ranged from .60 (Composite) to 1.51 scale score units (Science 
Reasoning). Females, however, typically had higher average ACT English scores than 
did males when PLAN score, family income, and majority/minority ethnic group 
membership were statistically controlled (median average score difference = .53). 

Although average score differences for males and females might seem fairly large, 
gender differences in performance were, in fact, reduced considerably when other 
background and course work variables were statistically controlled. As shown in Table 
3, when gender was included as the only independent variable in the ACT Mathematics 
model (i.e., PLAN score, background variables, and course work variables were not 
statistically controlled), the median regression coefficient associated with this variable 
was relatively large (1.58 vs. .64). In other words, the typical ACT Mathematics mean 
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for males was 1.58 scale score units higher than that for females, when PLAN score, 
background variables, and course work variables were not statistically controlled. When 
these variables were statistically controlled, this difference decreased to .64 score units. 

Findings for the ACT Science Reasoning and Composite models, summarized in 
Table 3, indicated that gender differences in performance were similarly reduced by 
statistically controlling for PLAN score, background, and course work variables. The 
same is true for ACT English when statistically controlling for PLAN score and 
background. However, the reductions for these three models were much smaller than 
those for the Mathematics model. Findings for the Reading model, on the other hand, 
indicated a slight increase in average score differences by gender when PLAN score and 
background variables were statistically controlled. 

TABLE 3 

Median Regression Coefficients for Gender and 
Majority/Minority Ethnic Group Membership 



Median regression coefficients 



ACT Assessment score 


Gender only 


Gender, given course work 
taken 1 , majority/minority 
membership, family 
income, & PLAN score 


Ma j o ri ty/m inori ty 
membership only 


M a j o rity/min ori ty 
membership, given course 
work taken 1 , family 
income, & PLAN score 


English 


.57 


.53 


2.52 


.48 


Mathematics 


-1.58 


-.64 


2.14 


.42 


Reading 


-.07 


-.11 


2.81 


.36 


Science Reasoning 


-1.56 


-1.51 


2.48 


.67 


Composite 


-.67 


-.60 


2.42 


.48 



bourse work variables were not included in ACT English and Reading models. 
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Ethnicity and income. Caucasian-American students typically had higher mean 
ACT scores than did ethnic minority students when PLAN score, course work (ACT 
Mathematics, Science Reasoning, and Composite models only), gender, and family 
income were statistically controlled. The median differences in average score ranged 
from .36 (Reading) to .67 (Science Reasoning) scale score units. When majority /minority 
ethnic group membership was used singly to model ACT Assessment scores, typical 
average ACT score differences were considerably larger, ranging from 2.14 (Mathematics; 
see Table 3) to 2.81 (Reading) scale score units. This suggests that the other variables 
included in the models played a significant role in diminishing the effects of differential 
ACT Assessment performance between majority and minority ethnic groups. 

Family income level contributed very little to average ACT performance 
differences when PLAN score, course work, gender, and majority /minority ethnic group 
membership differences were statistically controlled. Typical average ACT score 
differences between family income levels did not exceed .08 scale score units in any of 
the models. 

Regression Statistics by School Performance Category 

Tables 4a-4c show distributions, across schools, of regression statistics by school 
performance category. Each table corresponds to a different ACT test score model 
(Mathematics, Science Reasoning, Composite). Regression statistics by school 
performance category were not developed for the English and Reading models because 
course work variables (the only variables directly relevant to school attended) were not 
included in these models. There were 125, 664, and 90 schools, respectively, in the 
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categories of performance below expectation, performance as expected, and performance 
above expectation. 



TABLE 4a 

Distributions, Across Schools, of Regression Statistics, 
for ACT Mathematics, by School Performance Category 



School performance category 





Below expectation 
(k=125) 


As expected 
(k=664) 


Above expectation 
(k=90) 


Statistic 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


Med. 


Min. 


Max. 


N 1 


43 


25 


182 


46 


25 


380 


58 


27 


228 


R 


.84 


.55 


.95 


.87 


.69 


.98 


.88 


.78 


.95 


SEE 


2.33 


1.11 


3.46 


2.45 


1.18 


3.58 


2.47 


1.65 


3.58 


Intercept 


5.91 


-4.16 


13.38 


3.56 


-7.61 


17.27 


5.07 


-9.37 


18.70 


Regression coefficients 




















PLAN Mathematics 


.67 


.12 


1.09 


.82 


.16 


1.53 


.80 


.15 


1.11 


Trigonometry 


1.24 


-1.97 


5.50 


1.33 


-4.41 


7.42 


1.50 


-3.36 


4.98 


Calculus 


1.26 


-5.10 


5.75 


1.36 


-7.34 


10.50 


.96 


-4.74 


4.07 


Chemistry 


.72 


-3.31 


5.21 


.88 


-5.21 


7.92 


1.17 


-8.07 


7.12 


Physics 


1.19 


-1.92 


4.83 


.92 


-5.57 


6.87 


1.03 


-2.89 


5.43 


Gender 


-.70 


-3.71 


1.53 


-.65 


-4.91 


2.91 


-.67 


-2.62 


1.82 


Family income 


.02 


-.48 


.56 


.03 


-.69 


.90 


.05 


-.34 


.89 


Majority /minority 


.36 


-3.34 


4.60 


.47 


-8.24 


8.72 


.29 


-8.02 


5.84 



*N = number of students, k = number of schools. 
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TABLE 4b 

Distributions, Across Schools, of Regression Statistics, 
for ACT Science Reasoning, by School Performance Category 



School performance category 



Statistic 


Below expectation 
(k=125) 

Med. Min. Max. 


As expected 
(k=664) 

Med. Min. Max. 


Above expectation 
(k=90) 

Med. Min. Max. 


N 1 


43 


25 


182 


46 


25 


380 


58 


27 


228 


R 


.80 


.42 


.93 


.82 


.57 


.95 


.82 


.68 


.92 


SEE 


2.59 


1.16 


3.70 


2.71 


1.61 


3.83 


2.78 


2.11 


4.15 


Intercept 


4.06 


-4.52 


14.91 


3.25 


-8.62 


16.47 


3.07 


-8.55 


13.91 


Regression coefficients 




















PLAN Composite 


.81 


.21 


1.54 


.94 


.28 


1.68 


.92 


.57 


1.36 


Trigonometry 


.43 


-5.36 


4.59 


.38 


-6.59 


4.68 


.52 


-3.85 


4.33 


Calculus 


.03 


-6.12 


6.50 


.16 


-8.91 


7.52 


.47 


-5.51 


4.99 


Chemistry 


.45 


-3.63 


4.87 


.69 


-8.15 


8.17 


.75 


-5.69 


5.74 


Physics 


.41 


-3.85 


5.23 


.58 


-8.45 


8.60 


.62 


-2.96 


5.20 


Gender 


-1.15 


-4.69 


2.06 


-1.53 


-4.86 


2.45 


-1.62 


-4.20 


2.42 


Family income 


.03 


-.58 


.64 


.01 


-.72 


.59 


.04 


-.45 


.63 


Majority / minority 


.77 


-4.99 


10.17 


.64 


-6.77 


10.42 


.92 


-4.28 


11.93 



'N = number of students, k = number of schools. 
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TABLE 4c 

Distributions, Across Schools, of Regression Statistics, 
for ACT Composite, by School Performance Category 



School performance category 



Statistic 


Below expectation 
(k=125) 

Med. Min. Max. 


As expected 
(k=664) 

Med. Min. Max. 


Above expectation 
(k=90) 

Med. Min. Max. 


N 1 


43 


25 


182 


46 


25 


380 


58 


27 


228 


R 


.89 


.70 


.97 


.91 


.63 


.97 


.91 


.75 


.96 


SEE 


1.90 


1.18 


3.25 


1.96 


1.15 


3.12 


1.92 


1.47 


2.57 


Intercept 


.79 


-6.79 


6.08 


.50 


-9.69 


10.89 


1.84 


-10.21 


10.48 


Regression coefficients 




















PLAN Composite 


1.02 


.57 


1.36 


1.06 


.57 


1.45 


1.01 


.64 


1.34 


Trigonometry 


.43 


-2.82 


4.24 


.62 


-3.26 


3.73 


.69 


-2.56 


3.18 


Calculus 


.46 


-4.03 


4.92 


.51 


-5.53 


5.07 


.56 


-3.96 


3.77 


Chemistry 


.38 


-3.57 


3.98 


.65 


-5.03 


5.84 


.56 


-2.71 


4.54 


Physics 


.66 


-2.50 


4.51 


.51 


-3.06 


7.23 


.38 


-2.39 


3.73 


Gender 


-.53 


-2.47 


2.10 


-.61 


-3.55 


2.06 


-.64 


-2.70 


1.45 


Family income 


.06 


-.47 


.51 


.03 


-.65 


.54 


.05 


-.31 


1.02 


Majority /minority 


.56 


-4.21 


5.90 


.49 


-5.72 


6.35 


.40 


-4.09 


7.14 



'N = number of students, k = number of schools. 
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Median average differences in ACT score for students taking and not taking the 
courses, represented by the median regression coefficients for the Mathematics, Science 
Reasoning, and Composite models, are plotted by school performance in Figure 1. The 
effect of taking trigonometry, calculus, chemistry, and physics differed across school 
performance categories for each model: In general, smaller average score differences 
between students taking and not taking a given course were associated with lower- 
performing schools, whereas larger average score differences were associated with 
higher-performing schools. Figure 1 illustrates, for example, that for schools categorized 
as performing below expectation, the median average ACT Mathematics score was 1.24 
scale score units higher for students taking trigonometry than for students not taking 
trigonometry. For schools categorized as performing as expected and performing above 
expectation, median average Mathematics score differences were somewhat larger (1.33 
and 1.50, respectively). 

The effect of taking trigonometry, calculus, chemistry, and physics was greatest 
for ACT Mathematics score, relative to ACT Science Reasoning and Composite scores. 
The first plot in Figure 1 shows that across school performance categories, average score 
differences in ACT Mathematics score ranged from .72 (chemistry) to 1.50 (trigonometry) 
scale score units when PLAN score, gender, family income, and majority /minority ethnic 
group membership were statistically controlled. The second and third plots in Figure 
1 show the results for Science Reasoning and Composite scores. On ACT Science 
Reasoning, average score differences typically ranged from .03 (calculus) to .75 
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(chemistry) scale score units. On the ACT Composite, average score differences ranged 
from .38 (chemistry and physics) to .69 (trigonometry) scale score units. 



FIGURE 1. Median Increase in ACT Score for Courses Taken, by School Category. 
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The effects of gender, ethnicity, and family income were also compared across 
school performance categories. In all cases, no differences were found across 
performance categories: Coefficients for these variables were comparable to those based 
on data summarized across schools (see Table 2a). 

Characteristics of Schools 

Table 5 contains distributions across schools of descriptive statistics, by school 
performance category. The results in this table show trends in student test performance 
and course taking across school performance categories. As might be expected, 
relatively low average PLAN and ACT Assessment performance was associated with 
students at lower-performing schools, whereas relatively high average test performance 
was associated with students at higher-performing schools. Similarly, relatively smaller 
median percentages of courses taken were associated with students at lower-performing 
schools; larger percentages occurred for students at higher-performing schools. 

Table 6 summarizes school characteristics across school performance categories. 
Of the schools categorized as performing below expectation, 85% were located in 
districts where 12% or more of students were below the poverty level. For schools 
performing as expected and schools performing above expectation, only 35% and 9%, 
respectively, were located in districts where 12% or more of students were below the 
poverty level. 

The district per-pupil expenditure of schools categorized as performing below 
expectation was lower than that of schools categorized as performing as expected. This 
expenditure, in turn, was lower than that of schools categorized as performing above 
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TABLE 5 

Distributions, Across Schools, of Descriptive Statistics for PLAN Scores, ACT 
Assessment Scores, and Independent Variables, by School Performance Category 



School performance category 



Statistic 


Variable 


Below expectation 
(1^=125) 

Med. Min. Max. 


Med. 


As expected 
(k=664) 

Min. 


Max. 


Above expectation 
(k=90) 

Med. Min. Max. 


Mean 


PLAN 

English 


17.7 


13.3 


19.9 


19.9 


14.9 


22.9 


21.9 


20.0 


24.2 




Mathematics 


17.1 


14.4 


19.2 


19.3 


16.2 


21.6 


21.4 


19.7 


22.8 




Reading 


16.9 


12.5 


19.0 


19.2 


15.7 


22.4 


21.1 


19.9 


23.4 




Science 


17.5 


15.1 


19.2 


19.3 


16.5 


21.9 


20.8 


19.8 


22.2 




Reasoning 

Composite 


17.5 


13.9 


18.8 


19.5 


16.8 


21.4 


21.4 


20.5 


23.1 




ACT Assessment 
English 


19.6 


15.5 


21.3 


22.3 


17.6 


24.7 


24.8 


23.6 


27.3 




Mathematics 


18.6 


16.0 


21.6 


21.6 


18.3 


25.0 


24.6 


23.0 


27.1 




Reading 


19.9 


15.4 


21.7 


23.0 


16.8 


26.2 


26.0 


24.3 


29.3 




Science 


19.6 


16.4 


21.3 


22.4 


17.0 


24.5 


24.7 


23.4 


27.0 




Reasoning 

Composite 


19.6 


16.1 


20.5 


22.5 


17.5 


24.6 


25.0 


24.5 


27.7 


Percent 


Course work taken 

Trigonometry 


41 


7 


97 


47 


3 


98 


63 


9 


94 




Calculus 


11 


2 


44 


12 


1 


55 


15 


2 


59 




Chemistry 


85 


26 


98 


86 


23 


99 


91 


28 


99 




Physics 


27 


6 


83 


28 


2 


96 


39 


8 


94 




Gender 

Female 


62 


33 


99 


59 


1 


99 


60 


37 


82 




Male 


38 


1 


67 


41 


1 


99 


40 


18 


63 




Family income 

Less than $11,999 


12 


3 


61 


4 


1 


48 


3 


1 


13 




12,000 - 23,999 


20 


3 


44 


10 


1 


41 


6 


1 


19 




24,000 - 35,999 


24 


7 


46 


19 


3 


48 


12 


2 


39 




36,000 - 49,999 


20 


2 


42 


23 


3 


48 


19 


3 


41 




50,000 or more 


21 


2 


64 


40 


3 


89 


60 


18 


88 




Majority /minority 
Majority 


l 73 


2 


98 


95 


4 


99 


97 


56 


99 




Minority 


27 


2 


98 


5 


1 


96 


3 


1 


44 



] k = number of schools. 
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TABLE 6 

Percentages for Selected Background Variables, 
by School Performance Category 



School performance category 

Below As Above 

expectation expected expectation 

Level Variable (k'=i25) (k=664) (k=90) 



School district 



Student 



Percent below poverty 
level 



0 - 4.9% 


i 


30 


52 


5 - 11.9% 


15 


35 


39 


12 - 24.9% 


54 


31 


8 


25% or more 


31 


4 


1 


Per-pupil expenditure 


$.00 - 3199.99 


32 


25 


12 


3200 - 4199.99 


41 


40 


29 


4200 - 5199.99 


12 


18 


18 


5200 or more 


15 


17 


42 


Location 


Rural 


47 


30 


15 


Suburban 


26 


43 


50 


Urban 


27 


27 


36 


Race /ethnicity 


African-American 


22 


5 


3 


Asian- American 


0 


0 


0 


Caucasian- 

American 


67 


91 


95 


Hispanic 


11 


5 


3 



'k = number of schools. 



expectation. The majority of schools categorized as performing below expectation were 
identified as having rural locations (47%), rather than suburban (26%) or urban (27%) 
locations. In comparison, the majority of schools categorized as performing as expected 
or performing above expectation were identified as having suburban locations (43% and 
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50%, respectively). At schools categorized as performing below expectation, relatively 
large percentages of students indicated that they were members of ethnic minority 
groups (33%). At schools performing above expectation, relatively large percentages of 
students indicated that they were Caucasian- American (95%). 

Discussion 

The results of this study indicate that, in a typical high school, students who take 
upper-level mathematics or science courses (e.g., trigonometry, calculus, chemistry, or 
physics) can expect, on average, to earn meaningfully higher ACT Mathematics, Science 
Reasoning, and Composite scores than students who do not take these courses. This is 
true regardless of students' gender, family income, majority /minority ethnic group 
membership and PLAN scores. Taking English and social studies courses has little, if 
any, effect on ACT English and Reading scores, while controlling for PLAN score. This 
does not imply that students receive no benefit from taking these courses, however; it 
merely suggests weaker content relationships for English and social studies courses, 
while controlling for PLAN score, relative to those observed for mathematics and science 
courses. 

The effects of taking mathematics and science courses varied across school 
performance categories. Generally, smaller mean differences in ACT score were 
associated with schools performing below expectation. This indicates that taking 
mathematics and science courses will differentially affect the achievement of higher- 
order thinking skills of students, as measured by ACT scores, depending on the school 
they attend. Schools in poorer districts with higher percentages of ACT-tested ethnic 
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minorities are more likely to show small mean differences than are other types of 
schools. These achievement differences are likely attributable to such factors as 
differences in instruction, course content, or instructional materials, although other 
factors could also account for them. 

The models examined in this study provide some insight into differences in ACT 
Assessment performance for different population subgroups. The results show that ACT 
performance differences, particularly on the Mathematics test, are reduced for males and 
females when PLAN score, course work taken, majority /minority membership, and 
family income are considered. Similarly, ACT score differences between Caucasian- 
Americans and ethnic minorities are considerably reduced when PLAN score, course 
work taken, gender, and family income are considered. It is likely that other important 
(i.e., non-cognitive) variables could reduce these differences further. 

Family income level contributed minimally to average ACT performance 
differences when controlling for PLAN score, course work taken, and background 
variables. This suggests that most of the sources of ACT performance differences among 
students with varying income backgrounds were identified in this study. 
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Appendix B 
TABLE B.l 



Descriptive Statistics for PLAN Scores, ACT Scores, and Independent Variables 
(Based on Data Pooled over High Schools 1 ) 







Statistic 




Variable 


Mean 


SD 


Percent 


PLAN 


English 


19.8 


4.4 


- 


Mathematics 


19.3 


3.8 


- 


Reading 


19.2 


4.6 


- 


Science Reasoning 


19.3 


3.2 


- 


Composite 


19.5 


3.4 


- 


ACT Assessment 


English 


22.3 


5.0 


- - 


Mathematics 


21.8 


4.9 




Reading 


23.0 


5.9 


- 


Science Reasoning 


22.4 


4.6 


- 


Composite 


22.5 


4.5 


- 


Course work taken 


Trigonometry 




- 


49 


Calculus 


- 


- 


13 


Chemistry 


- 


' - 


81 


Physics 




- 


33 


Gender 


Female 


- 


- 


58 


Male 


- 


- 


42 


Family income 


Less than $11,999 




** 


5 


12,000 - 23,999 


- 


- 


11 


24,000 - 35,999 


- 


- 


19 


36,000 - 49,999 


- 


- 


21 


50,000 or more 


- 


- 


45 


Majority/ minority 


Majority 


- 


- 


89 


Minority 


- 


- 


11 




'Total number of students = 73,818. 
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